"A SYSTEM FOR THE AUTOMATIC ANALYSIS OF IMAGES SUCH AS 
DNA MICROARRAY IMAGES" 

Field of the invention 

This invention relates to the sector of analysis of 
images and was developed with special reference to the 
possible application to DNA analysis, especially in 
view of the automatic analysis of the images generated 

^ so-called DNA microarray or DNA chip. 
The autTomatic analysis ol DNA is mainly based on the 
examination of the messenger RNA which controls the way 
in which the various parts of the genes are activated 
or deactivated to create certain types of cells. 
If the gene is expressed in a single way, it can 
generate a normal muscular cell, while if it is 
expressed in another way it can generate a tumour. 
By comparing the different expressions of genes, 
researchers aim at discovering the ways to predict and 
prevent cancer. 

Another possible application is the so-called 
pharmacogenomics, a discipline in which scientists 
attempt to correlate the smallest DNA variations of a 
person with reaction to various substances, such as 
drugs . 

Numerous other • possible applications are being 
implemented, developed and studied. 
I>e-eea-i-pLloTi ~o ' I Lli e— pirdrar ai. L - 

Over the past years, a technique based on the use of 
so-called DNA chips has been developed to allow 
automatic DNA analysis. 

Essentially, DNA chips are small flat surfaces on which 
some rows, called probes, of one half of the double 
helix of DNA, are deposited according to a typical 
matrix configuration. 

Since each half of the double helix of DNA is naturally 
bonded to its complementary other half in a process 



called hybridisation, the DNA chip can be used to 
identify the presence of particular genes in a 
biological specimen . 

These chips are called microarrays in relation to their 
matrix structure, which may also be linear, and can be 
made employing different technologies, including 
semiconductor technology, on a variety of surfaces, 
including glass and plastic. 

The use of DNA microarrays to delineate the expression 
of genes is the most important application of 
"biochips". This method has completely replaced the 
previous methods which had the disadvantage of needing 
to be repeated either on each gene or on a restricted 
number of genes and were also difficult to automate. 
For a general illustration of a possible application of 
these methods, useful reference can be made to the work 
by DeRisi J et . al . , "Use of cDNA microarray to analyse 
gene expression patterns in human cancer", NatGenet 
1996 Dec; 14 (4), 457-60. 

Usually, DNA microarrays are used as interconnected 
memory chips in order to compare specimens of DNA from 
a patient against known, preserved specimens of DNA. 
This is because DNA carries an electrical charge and 
this charge can be read on a chip, exactly in the way 
that occurs in a cell in a matrix of memory cells. 
In many DNA chips, the coupling of arrays of DNA is 
signalled by means of fluorescent materials. 
Notwithstanding, the procedure for analysing the chip, 
in particular to detect the levels of fluorescence, is 
rather costly. 

Various methods have been developed to avoid these 
problems. . 

For example, according to a known solution, developed 
by the company Micro Sensors in collaboration with the 
company Motorola, DNA probe coupling is detected by 
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means of bio-electronic methods. 

This solution essentially consists in depositing a 
number from 10 to 50 DNA probes on a printed circuit. 
An organic atom containing iron which can generate an 
electronic signal when the DNA rows are coupled, is 
used instead of fluorescence. 

Parallel methods, allowing the simultaneous 

quantification of the level of expression of a very 
high number of genes by means of simultaneous querying, 
with a high sensitivity and fidelity of acknowledgement 
of the expression profile of a complete library of 
□ genes, have been developed over the past years. 

^ With a certain degree of approximation, yet essentially 

fU close to reality, the methods based on the use of 

15 microarrays of genes can be ideally related to some 
main classes. 
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W The method deve]\^ped by Prof. Brown represents a first 

!™ class of solutions. This method permits, by means of 

^ robot micro-machining, to chemically immobilise in 2 by 

'r^ 20 2 cm micro-grids fragments of cDNA (complementary DNA) , 

p or DNA reconstructed on the basis of RNA by reverse 

M transcription. In tnis way, microarrays containing 

10,000 individual cDNA elements are formed. The DNA 
fragment to be analysed is marked with fluorescent 
^-<^>y\ groups so to obtain different types of sensors to 

immediately distinguish the fragments of DNA by means 
of the colour of the cof\responding fluorescent group 
with which they are treated. In this way, the 
microarray can be analysedX simultaneously during the 
hybridisation phase. The micro-grid is read by means of 
a confocal microscope at theXend of the hybridisation 
phase providing a two-dimensional image in which 
coloured pins, or spots, appearXarranged in a grid. The 
intensity of the various colours \and their combinations 
is directly correlated to the in^tensity of the light 
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output by fluorescence by the respective probes and to 
the degree of affinity between the probes and the 
individual genes deposited on the grid. 

Another technique is known as micro-spotting. In this 
5 technique, a robot arm is dipped in a DNA material in 
correspondence with an array of pins which is then 
impressed on a glass support. 

Another method based on the use of microarrays was 
introduced by Affymatrix. This technique employs 
10 synthetic oligonucleotides, instead of natural 
fragments of DNA for constructing the micro-grid. These 
g fragments are deposited on the grid by means of 

%0 photolithography. In particular, masks for exposing 

™ some parts of a glass wafer on which certain chemical 

-3 15 processes occur are used to make single row DNA 

^ sensors . 

Q The use of photolithography, in combination with the 

chemical synthesis of oligonucleotides, results in a 

P 

^ presence of approximately 100,000 genes in a single 

^ 20 microarray which, according to current estimates, 

J— 

S compose the complete library of mapped human 

U. characteristics . 

The methods described provide as a final result an 
image which expresses the degree of genie expression in 
a fragment of DNA to be analysed by means of shades of 
different colours or combinations of colours. 
The main advantage of the microarray method consists in 
the possibility of simultaneously analysing an 
extremely high number of genes. 

This is because the information associated with the 
different cells present on the DNA chip can be 
processed in parallel with the consequent possibility 
of increasing the number of cells in the microarray to 
values in the order of 10,000 - 100,000 cells. 
In this way, systems for the automated analysis of 
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fragments of DNA can be provided which employ 
processing techniques of the images derived from 
microarray after hybridisation. 

This notwithstanding, the systems of this type 
implemented to date are based on the analysis of very 
large images (a number of pixels which is one to two 
orders of magnitude greater than the number of cells 
which form the micro-grid) . These images can be 
acquired in parallel, but are transferred and processed 
in a sequential fashion, as usually occurs in analysis 
techniques employing digital microprocessors, whereby 
processing speed is considerably penalised. 
Consequently, the idea of using a DNA chip has not been 
fully exploited to date, due to the 'difficulty in 
achieving real time analysis of the respective 
fluorescent images. Moreover, since diagnostic 
protocols generally require a certain number of 
microarray-based experiments, the time required for 
analysing the resulting images (processing times in the 
range of 10 to 3 0 minutes) abnormally hinder the 
efficacy of such method. 

The need therefore exists to provide an alternative 
system able to process images in real time, such as the 
images generated by a microarray of the type described. 
The object of this invention is to provide a system 
which allows efficient, rapid automatic analysis of 
images, such as the images generated by a DNA. chip 
after hybridisation, to identify the affinities between 
the analysed specimen and the fragments of DNA on the 
DNA chip. 

According to this invention, this object is attained 
thanks to a system having Vhe characteristics which 
are specifically called for iVi the claims which follow. 



In essence, accordirto to the currently preferred 
embodiment, the invention provides for making a system 
which allows to automatically analyse the images from a 
DNA chip after hybridis^^t ion . This is attained by 
acquiring the images by means of optical matrix sensors 
and by processing the acquired images by means of a 
Cellular Neural Network (abbreviated to CNN) . Such a 
processing is essentially analogue and is achieved 
spatially on the entire development of the microarray 
matrix. \ 

For a general illustration of the characteristics of a 
cellular neural network, useful reference can be made 
to document US-A-5 140 670. 

According to the Wrrently preferred embodiment of the 
invention, images ^re analysed by means of a computing 
process which accoufy^s for the physical -chemical rules 
at the basis of reactions on the microarray. 
In a particularly advantageous way, the cellular neural 
network architecture comprises a matrix of cells which 
are locally interconnected by means of synaptic 
connections, the matrix presenting a spatial 
distribution which is essentially correlated to the 
matrix form of the processed images. 

A system according to the invention can be easily made 
according to a system-on-a-chip configuration, in which 
the entire acquisition and processing system of the 
images is integrated on a single chip, for example 
implementing VLSI CMOS technologies. Reference to this 
matter can be found in the work by Rodriguez -Vasquez A. 
et al . , "Review of CMOS Implementations of the CNN 
Universal Machine -Type Visual Microprocessors" 
published in Proceedings of ISCAS 2000 (IEEE Int. 
Symposium on Circuits and Systems), Geneva, May 28-31, 
2000 . 

More in particular, this iWention relates to a system 



integrated in a morfolithic fashion on a semiconductor 
for automatically analysing images, such as images from 
a microarray of the \ type comprising optical matrix 
sensors for the acqui^tion of images and to a high 
computing power parallel analogue processing 
architecture, based on tkie implementation of cellular 
neural network. Moreover). the invention allows to 
integrate the entire imageX acquisition and processing 
system on a single chip. 
Characteristics and advantages of this invention will 
be illustrated with reference to a preferred 
embodiment, as non- limiting examples, in the enclosed 
drawings . 

Brief description of the drawings 

In the enclosed draw^gs which are attached to this 
description : 

- figure 1 illustrate>s a basic diagram of the 
embodiment of a system for automatically analysing 
images from a DNA chip a^ter hybridisation; 

- figures from 2 to 5 illiistrate, both in the form of 
block diagrams and in Vhe form of charts, the 
criteria on which the organisation and operation of 
the cellular neural network \are based; 

- figure 6 illustrates an example of image from a DNA 
chip after hybridisation and \spli tt ing thereof into 
three chromatic components; \ 

- figure 7 illustrates a flow chart which outlines the 
method whereby a cellular ne\ral network image 
processing sequence is applied \^o each chromatic 
component of an image read from a DNA chip; 

- figure 8 (which is split into two parts, identified 



by 8a and 8b, respectively) and figur^es from 9 to 12 



A 

illustrate, for example, the methodV according to 
which the various operations concerning filtering, 
segmenting, and the morphological operations, which 




^^^^ can be implemented in a system according to this 
Z^>^ invention, can be conducted to isolate useful 
information with respect to the various sources of 
noise which could lead to false interpretations of 
5 the results during the automatic microarray .image 
^ analysis process. 
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As mentioned above, \the solution according to this 
invention offers an \advantageous alternative with 
respect to traditional ntethods based on the analysis of 
fluorescence images genei^ted by means of a DNA chip. 
In particular, the solutidh according to this invention 
utilises the class of\ arrays (generally two- 
dimensional) of analogue processors known as cellular 
neural networks (CNN) and img)lements a system which is 
able to process such images iiAreal time. 

Reference I in figure 1 indicates an image, for example 
in the form of a square matrix of spots on a DNA chip 
(of the known type and, consequently, not illustrated 
in the figures) . 

The image is "read", preserving the matrix 
organisation, by an optical sensor made, for example, 
employing CMOS technology and associated with a 
processing system of the type shown in figure 2 and 
indicated in general by number 20. 

As further illustra&ed hereof, the system 20 can be 
configured as a 6ellular neural network (CNN) 
processing system, i \e . as an analogue, parallel 
processing system, preT^rably integrated in the same 
chip housing the block 10^ in which the optical sensor 
is integrated. 
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In particular (a^aain with reference to the block 
diagram in figureX 2) , in addition to the array of 
analogue cells withXoptical sensors forming the block 

10 in which the opXical sensor is integrated, the 
system preferably comprises a set of analogue memories 

11 which can co-operat^ with sensor 10, according to 
the criteria which are further described below, as well 
as an input/output circuit\l2, which type is generally 
known . 

The operation of components 10 and 12 described above 
is achieved under the supervision of a control logic 

Preferably, the control ^gic 13 directly acts on the 
circuit 12. The same control logic 13 is usually 
configured so to directly Operate on the array 10 by 
means of an analogue/digital Converter 14 to which the 
instructions contained in Va program memory 16 
selectively flow via a set of digital registers 15 for 
the configuration of the cellulalf neural network. 
According to another important Characteristic of the 
invention, the system 2 0 is configured as a cellular 
neural network which avoids the \need to implement 
analogue/digital conversion and/or Vice versa of the 
values of each element or pixel in t\ie image acquired 
at output of the optical sensor 10, \lso allowing to 
implement the microarray image analVsis algorithm 
according to a totally parallel processing criterion. 
The various operations forming the algorithm are 
achieved by Suitably setting the parameters which are 
programmed in the configuration registers \l5 of the 
neural network on a case-by-case basis. \ 
In this way, an algorithm strictly speaking is created, 
being the sequence of elementary operations performed 
on the colour image, i.e. on its chromatic elements, 
composed preferably by the red component R and the 
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green component G only, i.e. with the exclusion of the 
blue component B, as shown in figure 1. 

Figures from 3 to 5 illustrate the principle 
implementing the model of a cellular neural network as 
the array of cells 100. The cells are reciprocally 
identical and only locally interconnected by means of 
weighed synaptic connections. 

The circuit model of ^ch cell 100 is shown in the 
diagram in figure 4, which schematically illustrates 
the values included in matrixes A and B and in the bias 
coefficient I. Said valuers allow to generate, from a 
input signal, a corresponding output value which is 
weighed by means of function h(x) illustrated in figure 

\ 

This all corresponds to known criteria which 
consequently do not need to be additionally illustrated 
herein . 

Returning to the blo^^ diagram in figure 2, the block 
10 essentially consists of a matrix of analogue cells 
whose inputs are the\ signals corresponding to the 
optical sensors which rd^d the image I generated in the 
microarray. \ 

The analogue memory 11 is\ used to store the images and 
the intermediate processing stages. Conversely, the 
instructions and the respective parameters are stored 
in digital form in the memory 16 and in the registers 
15 and are applied to the block 10 by means of the 
converter 14. The control 16gic 13 synchronises the 
image acquisition and prodessing operations, in 
addition to the I/O signals to Vhe end user which pass 
through the block 12 . . \ 

According to Chua and Yang, the model equations of a 
cellular neural network are: 
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m 



RCdxij/dt= -Xij + ZA(l,m).yin, + EB(l,m).uin, + Ibias 

where the sums extend to all values (l,m) belonging to 
the cells of the neighbourhood N(Cij) of the cell 
concerned Cij and 
yij= -1 if Xij < xiow 

1 if Xij > Xhigh 

Xij in other cases 
A possible differentiation of the model, known under 
the name of FSR range (Full State Range) model, is 
related to circuit simplification when implementing the 
circuit with VLSI CMOS technology: 



flj Tdxij/dt- -g(Xij) + SA(l,m).yi^ + ZB ( 1 , m) . ui^, + Ibia 
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where, also in this case, the sums extend to all values 
UJ (l,m) belonging to N(Cij) and 

L g(Xij) = Xiow if Xij < X|ow 

Q Xhigh if Xij > Xhigh 

20 0 in other cases 

p Obviously, the augorithms to be implemented depend on 

M the type of analy^sis required by the expert. However, 

important steps, \ such as the reduction of the 
components, noise \clearing, or the elimination of 
\ 25 deformed spots, willXneed to be performed in any case. 

The example shows an\ algorithm which extracts from an 
image resulting from \two red and green fluorescence 
probes the spots relatW to three different levels of 
each colour indicating the three different degrees of 
affinity between the prdbes and the genes present in 
the micro -grid . 
Figure 6 illustrat^ an example of image I from a DNA 
^^^;^v^C>\^ chip after hybridisation. For classification of 
affinities, analysing \the two chromatic components R 
(red) and G (green) onlyv will usually suffice. This is 
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because there are rto reactions able to generate 
appreciable levels of tVie component B (blue), i.e. the 
third component of thA known RGB (Red Green Blue) 
colour model . \ 

This implies firstly a possible hardware simplification 
of the image acquisition part, i.e. of the optical 
sensor comprised in the block 10. 

As mentioned, optical sensors are used for reading DNA 
chip images (for example CMOS) . The optical sensors can 
be either black and white sensors or Bayes four-section 
RGGB sensors. The resulting image is converted, .after 
digitalisation, into an RGB, YUV image, etc., according 
to the type of processing and the reference 
application . 

In the solution accdrding to this invention, this form 
of pre-processing ca\i be eliminated and simple two- 
colour sensors, instea^d of Bayes sensors, can be used 
as sensors which are selectively sensitive to distinct 
chromatic components. \ 

Furthermore, no \digitalisation operation is required, 
since a typicallV analogue treatment is implemented. 
Consequently, applving an image processing sequence 
with cellular neurad networks based on templates, for 
example according tdv the process outlined in the flow 
chart in figure 7, foir each of the chromatic components 
(R and G) processed, woi^l suffice. 

The reader will certainly appreciate that the sequence 
of processing operations illustrated herein is adapted 
to be implemented fully in parallel, • i.e. 
simultaneously (with a consequent reduction of total 
processing times) on the two chromatic components R and 
G. These latter are . derived from the source image I in 
a known way, for example by filtering or by exploiting 
the availability of optical sensors with chromatically 
selective characteristics . 
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In essence, according to the currently preferred 
embodiment of this invention, the processing sequence 
comprises : 

1. a background clearing operation, implemented in 
steps indicated by the numbers 201 and 301; 

2. a grid analysis operation, implemented in steps 
indicated by the numbers 202 and 302; 

3 . an operation for eliminating the smaller irregular 
spots, implemented in steps indicated by the numbers 
203 and 303; 

4 . an operation for eliminating the larger spots, 
implemented in steps indicated by the numbers 204 
and 3 04; 

5. an intensity analysis operation, implemented in 
steps indicated by the numbers 205 and 305; 

6. a thresholding operation, for example on three 
levels, implemented in steps indicated by the 
nunobers 206 and 306 and finally 

7. a result combination operation in relation to the 
two analysed chromatic components implemented, for 
example, by means of a logical product (AND) in a 
final step indicated by the number 40. 

The three levels (high, medium and low) according to 
which the threshold definition operation indicated by 
blocks 206 and 306 is carried out are respectively 
indicated . by the numbers 2061, 2062 and 2063 (red 
component R) and by the numbers 3061, 3062 and 3063 
(green component G) . 

All the operations above ,VLncluding the final logic AND 
operation, are carried outV within the cellular neural 
network by means of templates, i.e. by means of 
suitable sets of parameters wfvlch are programmed in the 
network configuration register^ (indicated by number 15 
in the diagram in figure 2) on a\case-by-case basis. 
The sequence of operations gives rise to a set of 
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intermediate results corresponding to images which can 
be stored in the analogue memory of the system, 
indicated by number 11 in figure 2. 

Figures 8 and 12 indicate, for example, the 
intermediate results corresponding to the main 
operations where certain specific operations involving 
filtering and segmenting and morphological operations 
are required in order to isolate the sources of noise 
which could lead to false interpretations of results 
during automatic analysis of the image I obtained from 
the microarray. 

More in detail, figure 8, which is split into two 
parts, identified by 8a and 8b, respectively, refers to 
the background clearing operation (steps 201 and 301 in 
the chart in figure 7) . 

In a first solution, illustrated in figure 8a, the 
source image to be processed, indicated by number 50, 
consisting of a set of spots, is subjected to 
thresholding operation with respect to a fixed value 
(for example a threshold equal to 0.85 of the maximum 
normalised intensity value of the image) to obtain the 
resulting image 51. 

In the variant illustrated in figure 8b, the same 
result is attained by diffusion filtering, shown by 
number 52, based on the implementation of templates 
which put the spot contours out of focus. An averaging 
operation, shown by number 53, is carried out on the 
image thus obtained to form the resulting image, which 
also in this case is indicated by the number 51. 
In the case in which the image processed (here supposed 
to coincide with the image 51 seen above, which is not 
imperative) is "dirty", for example, as shown in figure 
9, for the presence of a dot which does allow to 
identify the spots, an additional template or grid 55 
is used. Its function is to filter out the noise and 



eliminate the spots which overlap the contours of the 
grid 55. The resulting image is indicated by the number 
56 . 

Figure 10 and 11 illustrate the processing sequence 
attained by means of two other templates. 

In particular, figure 10 illustrates the application to 
a source image (here supposed to coincide with image 
56, which again is not imperative) of an erosion 
template which can erode the spots of said image on the 
right-hand side 57a, on the left-hand side 57b, in the 
horizontal direction 57c and in the vertical direction 
57d. 

In this way, the shape of the spots is analysed to 
eliminate the irregularities of the spots * by selecting 
only the largest circular spots. 

Figure 11, on the other hand, illustrates the sequence 
to implement direct intensity analysis to provide a 
classification of the spots in the source image 
(supposed to coincide with image 58 obtained above, 
which again is not imperative) on the basis of 
intensity. This occurs according to three threshold 
levels (for example equal to -0.5; 0 and +0.5; said 
threshold levels being referred to the maximum 
normalised intensity . 

The overall result which can be obtained is the 
generation of three images deriving from the threshold 
definition (and, consequently, of an essentially binary 
content, i.e. "dark" or "light" for each spot) 
indicated by the numbers 59a, 59b and 59c respectively, 
which can be used for the logic product operation 
(AND), indicated by block 40 in figure 7. 

This operation is schematically illustrated in figure 
12. Here, references 591 and 592 indicate, in general, 
two threshold images which are obtained respectively 
for the red component R and for the green component G, 
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combined by means of the logic product (AND) , to 
generate a final image 60 which can be made available 
to an end user in the form of a display (on screen 
and/or hard copy) driven by unit 12 in figure 2. 
5 The computing time requirements for each of the 
operations listed above shows a computing capacity 
which is much higher than the normal image processing 
techniques based on digital computer platforms. For 
example, by employing the chip time constant tcNN as a 
10 unit of time (which is typically in the order of 250 
nanoseconds) , each of ' the various template 
p implementation operations described above require 

typically from 3 to 6 of said units of time, which 
ly values which fall to one only of said unit in the case 

g 15 of simple logic operators and slightly higher times 

fj (for example, 10 tcNN units) in the case of recall 

UJ operations. 

1, In particular, the entire algorithm described above can 

m be run in approximately 275 microseconds, i.e. in less 

20 than 1 millisecond. 
P Varxous main advantages derive from the implementation 

M- of the solution according to this invention. 

Firstly, DNA can be analysed automatically and, 
consequently, objectively. This contrasts with a 
25 subjective analysis carried out by human operator 
employing normal digital image processing tools. 
The second advantage ih the high processing speed which 
allows to process images which can also be large 
directly on-chip with veVy short processing times. Such 

^-'^^^^X-^^ times depend only on the\ value of the time constant RC 

n:—^ ^ of the cells in the cellular neural network and the 



acquisition time of the \optical sensors because 



no 



analogue/digital conversion (and/or vice versa) is 
required for the values of^ each pixel of the image 
35 acquired at optical sensor output with respect to the 
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processing matrix opefNating in parallel with implements 
the analysis algorithm >of the microarray image. 
Finally, the system can easily be reprogrammed by means 
of a restricted number of coefficients which define the 
templates in the cellular neural network, corresponding 
to the single operations stored in the internal system 
memory in correspondence to values of the synaptic 
bindings of the adjacent cells. 

Naturally, numerous changes can be implemented to the 
construction and embodiments of the invention herein 
envisaged, all comprised within the context of the 
concept characterising this invention, as defined by 
the following claims. This especially refers to the 
possibility of applying the solution according to this 
invention to the processing of images of any nature. 
The scope of this invention is , therefore, not 
necessarily restricted to the processing of DMA chip 
images . 
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